Practical Optional Types for Clojure

Typed Clojure is an optional type system for Clojure, a dynamic language in the Lisp family that targets the JVM. Typed Clojure’s type system build on the design of Typed Racket, repurposing in particular occurrence typing, an approach to statically reasoning about predicate tests. However, in adapting the type system to Clojure, changes and extensions are required to accommodate additional language features and idioms used by Clojure programmers. In this paper, we describe Typed Clojure and present these type system extensions, focusing on three features widely used in Clojure. First, Java interoperability is central to Clojure’s mission but introduces challenges such as ubiquitous null; Typed Clojure handles Java interoperability while ensuring the absence of null-pointer exceptions in typed programs. Second, Clojure programmers idiomatically use immutable dictionaries for data structures; Typed Clojure handles this in the type system with multiple forms of heterogeneous dictionary types. Third, multimethods provide extensible operations, and their Clojure semantics turns out to have a surprising synergy with the underlying occurrence typing framework. We provide a formal model of the Typed Clojure type system incorporating these and other features, with a proof of soundness. Additionally, Typed Clojure is now in use by numerous corporations and developers working with Clojure, and we report on experience with the system and its lessons for the future. 1. Clojure with static typing The popularity of dynamically-typed languages in software development, combined with a recognition that types often improve programmer productivity, software reliability, and performance, has led to the recent development of a wide variety of optional and gradual type systems aimed at checking existing programs written in existing languages. These include Microsoft’s TypeScript for JavaScript, Facebook’s Hack for PHP and Flow for JavaScript, and MyPy for Python among the optional systems, and Typed Racket, Reticulated Python, and GradualTalk among gradually-typed systems. One key lesson of these systems, indeed a lesson known to early developers of optional type systems such as StrongTalk, is that type systems for existing languages must be designed to work with the features and idioms of the target language. Often this takes the form of a core language, be it of functions or classes and objects, together with extensions to handle distinctive language features. We synthesize these lessons to present Typed Clojure, an optional type system for Clojure. Typed Clojure builds on the core type checking approach of Typed Racket, an existing gradual type system for Racket. However, Typed Clojure extends this basic framework in multiple ways to accommodate the unique idioms 1 We reserve the term “gradual typing” for systems such as Typed Racket which soundly interoperate between typed and untyped code; systems like Typed Clojure or TypeScript which do not enforce type invariants we describe as “optionally typed”. (ann parent [’{:file (U nil File)} -> (U nil Str)]) (defn parent [{^File f :file}] (if f (.getParent f) nil)) Figure 1. A simple Typed Clojure program and features of Clojure, producing an expressive synthesis of ideas and demonstrating a surprising coincidence between multiple dispatch in Clojure and Typed Racket’s occurrence typing framework. The essence of Typed Clojure, of course, is Clojure, a dynamically typed language in the Lisp family built to run on the Java Virtual Machine (JVM) which has recently gained popularity as an alternative JVM language. It offers the flexibility of a Lisp dialect, including macros, emphasizes a functional style via a standard library of immutable data structures, and provides interoperability with existing Java code, allowing programmers to use existing Java libraries without leaving Clojure. Since its initial release in 2007, Clojure has been widely adopted for “backend” development in places where its support for parallelism, functional programming, and Lisp-influenced abstraction is desired on the JVM. As a result, it now has an extensive base of existing untyped programs, whose developers can now benefit from Typed Clojure. As a result, Typed Clojure is used in industry, experience we discuss in this paper. Figure 1 presents a simple program demonstrating many aspects of our system, from simple type annotations to explicit handling of Java’s null (written nil) in interoperation, as well as an extended form of occurrence typing and Clojure’s type hints, which are central to Typed Clojure’s approach to interoperability. The parent function has the type [’{:file (U nil File)} -> (U nil Str)] which means that it takes a hash table whose :file key maps to either nil or a File, and it produces either nil or a String. The parent function uses the :file keyword as an accessor to get the file, checks that it isn’t nil, and then obtains the parent by making a Java method call. The annotation ^File f is a type hint on f, which instructs the Clojure compiler (running prior to Typed Clojure typechecking) to statically resolve the getParent call to File’s getParent method with signature String getParent();, rather than using reflection at runtime. In the remainder of this paper, we describe how Typed Clojure’s central innovations, including Java interoperability, multimethods, and heterogeneously-typed immutable maps, enable this example and many others. We begin with an example-driven presentation of the main type system features in Section 2. We then incrementally present a core calculus for Typed Clojure covering all of these features together in Section 3 and prove type soundness (Section 4). We then discuss the full implementation of Typed Clojure, dubbed core.typed, which extends the formal model in many ways, and the experience gained from its use in Section 5. Finally, we discuss related work and conclude.